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ABSTRACT 

An innovative approach which vill be used to evaluate 
one of the Experimental Schools Programs of the National Institute of 
Education (NIE) is described* The project. Southeast Alternatives 
(SEA) , is designed to test the notion that comprehensive educational 
change is superior to piecemeal change; it consists of five 
educational programs-a contemporary school, a continuous progress 
school, a free school^ an open school, and a comprehensive high 
school* Some general descriptive material on the overall project is 
provided, followed by more specific and detailed discussions of the 
two major components of the evaluation — the Anthropological Study, a 
case studies approach designed to describe the evolution of SEA as 
comprehensively as possible, and the Impact Study, devised to assess 
the effect of SEA upon the schools, the students and their parents, 
and the community. For each of the components discussions are 
presented dealing with the assessment strategy, the data to be 
collected, methodological considerations, dependability of the data, 
and the analyses to be performed* Some major limitations of the 
evaluative strategies are also discussed* (PB) 
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The evaluation of Southeast Alternatives (SEA) is but one small 
piece of a complex evaluation effort conducted by the Experimental 
Schools Program (ESP) of the National Institute of Education (NIE). 
Early in its first year of operation (1971), ESP funded three experi 
mental schools projects designed to test the notion that comprehensi 
change is superior to piecemeal change in education. These projects 
were located in Franklin Pierce, Washington; Berkeley, California and 
Minneapolis, Minnesota. The project in Minneapolis is the subject of 
this paper. 

Soon after the project award, a national competition was held to 
select an on-site but external evaluation team. ARIES Corporation 
was selected to conduct the evaluation in Minneapolis and began planning 
in July, 1971. 

This paper presents a description of Southeast Alternatives (SEA) 
and the design developed by ARIES to evaluate and document SEA over 
its five-year life cycle. Of specific interest may be the anthropolog- 
ical techniques employed in the documentation tasks. 



1 The project presented and reported herein is supported by a 
Grant from the National Institute of Education; Department of 
Health, Education and Welfare. However, the opinions expressed 
herein do not necessarily reflect the position or policy of 
the National Institute of Education, and no official endorsement 
should be inferred. 
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A GENERAL INTRODUCTION TO SOUTHEAST ALTERNATIVES* 



The first announcements about ESP found Minneapolis, and 
especially Southeast Minneapolis, a community of varying value systems 
incompatible with a single approach to education. This, coupled with 
the educational philosophy** of School Superintendent, John B. Daivs, 
provided a school system both ready for and capable of designing a 
radically different approach to schooling. Hence, the beginning of 
SEA • 

Southeast Alternatives is a fitting designation for the Experi- 
mental Schools Project of Minneapolis. Students, parents and teachers 
have the opportunity to select f^^om alternative educational programs 
distinctly different in instructional pattern and diversity of curriculum 
material. A quote from the final plan adopted by the Office of 
Education reads, 

"Choice-making by students, teachers and parents 
win become the basic way of life for all members 
of the Southeast Alternatives program . • 

This is the primary goal of Southeast Alternatives. 

Five major programs, three elementary, one secondary and one 
K-12, are offered as options. Each program is formulating and imple- 
menting its own design for schooling. The five programs are: The 
Contemporary School, The Continuous Progress School, The Open School, 
the Free School (levels K through 12), and a Comprehensive (7-12) 
High School . 

Another goal of Southeast Alternatives is to provide the opportunity 
for student, teacher and community involvement in the shaping of the 
schools, their objectives and their curricula. 

From the inception of SEA, interested and involved parents have 
been hired on a part-time basis to disseminate information and encourage 
community input regarding the design of the project and project components. 
Many residents have participated in discussions held throughout the 
community. In the Free School, students and parents exercised their 
right of involvement by selecting the faculty. 



* It should be noted that SEA is presently redesigning its program in 
preparation for the three-year period, 1973-76. Much is being revised 
but the general themes will probably remain intact. 

**See "The Superintendent of the Minneapolis Public Schools Speaks in 

Opposition to the Voucher Plan," November 24, 1970. In this document 

Dr. Davis endorses options for parents and students within the public 
school system. 



The SEA director, who serves as the chief administrator for the 
project schools, reports to the assistant and associate superintendents 
of schools and is a member of the Minneapolis Public Schools Executive 
Cabinet. The SEA director has within the administrative office several 
supportive components. They are: 

• Research and Evaluation, This component is responsible for 
providing formative and summative evaluation valuable to the progress 
of the project and its components, 

f Staff Development. This component has as its major task the 
identification of training needs and the development of the appropriate 
training activities to reduce these needs and, therefore, strengthen 
the staff. The staff development program has evolved into a teacher 
center. 

§ Student Support Services. This component is responsible for 
coordinating the services of the counselors, social workers, school 
nurses and school psychologists; convening and sharing interviews 
for hiring the above, coordinating a "people" record system, planning 
a project staff budget for student support services and coordinating 
the educational programming of Marshall-University High School 
students. 

f Public Information. This component is responsible for: 
publishing a community -wide newspaper and other publications, develo.)- 
ing informational presentations, establishing visitation and obser- 
vation programs for the visitors of SEA and publishing and interpreting 
relevant research data concerning the progress of SEA. 

• Budget Control* Consisting of a financial advisor, this 
component is responsible for establishing and maintaining realistic 
budgets. 

• Community Liaison. The Community Liaisons are Southeast 
parents working within the various components. Their responsibility 
is to provide information and opinions from the community to SEA 
personnel regarding the operations of the program. They assist in 
developing wider citizen participation, identifying corrmunity resources 
establishment of teacher cadre, development of SEA programs, staff 
development, and they support the University In the continuation of 
its Southeast programs. They also serve as a communication channel 
between SEA, parents and community. 

• Community Education. This component is responsible for 
recruiting and supervising the community school coordinators for the 
respective SEA schools. They plan with the coordinators the community 
school programs for children and adults (extra-curricular), and 
coordinate these programs within SEA. They are also responsible for 
extended day or late afternoon classes which are provided at most 
Southeast schools. At Marshall-University, the evening classes 
serve over 400 people. 
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Organization and staffing for the individual instructional 
components have also undergone some changes. Project coordinators 
have assisted the principals of the Continuous Progress and Open 
Schools. Cadre staff were appointed to serve on a rotating basis 
providing a variety of services to the component schools and staff. 
The elementary teacher cadre is comprised of specialists in art, 
mathematics, music, science, industrial arts, environmental education, 
language arts and creative dramatics. Secondary cadre teachers work 
within departments to develop educational models and alternatives to 
be tested in the program. 

Another addition to the elementary staffs have been the appointment 
of guidance counselors to the faculties of the Contemporary, Continuous 
Progress and Open Schools. Elementary counselors Work, in part, with 
students to aid them in improving self esteem and relations with others. 

The following is a brief description of each alternative: 

Contemporary School - While incorporating a variety of 
recent educational innovations the emphasis in the Contemporary 
School is on teacher directed and sequenced instruction organized 
by classes by grade level. A science center, a well equipped 
math center with a portable computer terminal, an integrated art,, 
music, and drama activity room, a media center, a pottery and 
ceramics workshop, a photographic darkroom, and an industrial arts 
program combine to make up the innovative programs of the 
Contemporary School. 

The University of Minnesota and the Contemporary School 
have combined resources for their mutual benefit. The departments 
of Elementary Education, Secondary Education, Child Development, 
Educational Psychology, Theater-Arts, and the School of Dentistry 
have all been active in the Contemporary School. 

Continuous Progress School - The basic premise of this 
alternative's program 1s that "children learn best by working at 
their own pace.*' Accordingly, children progress through the basic 
skill curriculum without regard to grade levels or other artificial 
barriers* Such a process is designed to benefit both the slow 
learners and the academically talented children, each working at 
his own rate receiving assistance when needed. 

During the morning hours at the Continuous Progress School, 
the sessions are concerned with the basic skills, i.e., language 
arts, math, and social studies instruction. In the afternoon 
sessions, children choose from among art, music, science, social 
studies, photography, cooking, sewing, or other programs. 

Open School - That children, when given the opportunity to 
plan their own activities within a rich and carefully planned 
environment, will learn not only basic skills but also the ability 
to take initiative for their own education is the major premise 
of the Open School. To accomplish this task, the Open School is 
designed to encourage the exploration of educational subjects and 
materials under the guidance of teachers and parents. 
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The Open School has two basic organizational modes - Model I 
and Model II. Model I actually is an open classroom situation, 
where children with a semi self-contained classroom may study 
basic skills and the various arts. For the Model I program, 
there are two teachers, each with a class of thirty students. 
They are responsible for the teaching of all academic subjects 
as well as the other learning experiences which are considered 
to be Important, Children may leave the classroom to take 
industrial arts, gym and to use the multi-media facilities. 
Within the classroom there are various learning areas; a language 
arts area, math area, science area, etc. 

The Model II program which has undergone three modifications, 
was originally established to serve approximately 250 students. 
In the original Model II, each classroom was a specialized learn- 
ing center. Model II students (K-6) could visit any of these 
centers "at will" and could work there on a project until comple- 
tion. For a richer educational experience, the children were 
encouraged to move among all centers often and participate 
accordingly. The learning centers then established were: 

• Flair - This was the Reading Skills, Language Arts area 
consisting of three classrooms and a team of teachers available 

to help those students in the fundamental skills of language arts. 

• Systems Center - This Math area allowed students to study 
various ways that math is used* The room was further subdivided 
into areas for multiplication, addition, subtraction, and division; 
a time learning center; and a mathematical game area. 

t Social Studies Center - This area provided the children 
with the opportunity to confront the problems of their environ- 
ment and their society. 

0 Yellow Submarine - This was an art area. Skills obtained 
here included painting, sketching, weaving, sewing, ceramics, etc. 

e Inquiry Center (I*Q. Lab) - This IQ Lab encouraged the 
student to ask and answer questions about the scientific world. 
Here, students could pursue studies in chemistry, electricity, 
biology (live animals available), and astronomy. 

0 Hairmer Hall - This industrial arts area is the newest 
area for this building. Here, boys and girls can test their 
skills in the construction of wood and metal projects* 

9 Music Center - Under the supervision of its own music 
instructor, the students at Marcy may take singing, dancing, 
musical instruments, and/or study music appreciation. 

In Oecernber, 1971, Model II was revised to consist of four class- 
room units of "families" of 60 students each. These units contained 
elements of the former learning centers, which with the exception of 
industrial arts and music, were dismantled. 
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The Open School program is enhenced by a variety of support 
groups, Some two dozen parents and volunteers contribute services 
weekly to various school programs. Project 60, a University Department 
of Education project, provides juniors for support services within 
the component- A Marcy Advisory Council was established to serve in 
an advisory role to the principal. It has elected membership including 
parents, teachers, students, para-professionals and the other community 
members . 

At the beginning of the 1972-73 school year, the Open School 
again revised its structure, in part based on the finding of an in- 
house research unit. The revised model is briefly discussed in the 
following outline. 

• Age grouping: The K-6 "family" of 1971-72 has been 
modified into two more distinct age groupings, roughly 5-8 
years (primary) and 9-12 years (intermediate). Pairs of such 
"sub families" meet in rooms across a common hall from one 
another with a teacher in each room. These age sub-groupings 
are thought by the staff to allow staff to focus on interests 
and skills which correspond to the particular developmental 
stages of the children. The age division is not rigid, however, 
and occasions for a wider age mix are still provided. Students 
may spend time in both rooms, and this occurs especially among 
the 8 and 9 year olds. Students are not "graduated" from the 
primary to the intermediate room, but rather begin to spend 
more and more time with the atmosphere, activities, students, 
and teacher in the new surroundings. 

0 Time Parameters: The school day at Marcy School has 
become somewhat more structured over the two years of operation. 
The goals of having teachers forfeit (as much as possible) their 
authoritarian roles and having students become self-directive 
in their school activities are still important at Marcy. Indeed, 
it is thought by the staff that, rather than being obstructive, 
certain structural time parameters are valuably supportive to 
students as they attempt the difficult business of self-directed 
and profitable activity. 

9 Free School - Located in the United Campus Ministries 
Building of the University of Minnesota, this K-12 program 
offers the student "freedom" to pursue and study whatever is 
beneficial to his perception of himself, others and the 
environment. A first year student enrollment of 70 has expanded 
to 184 during the second year. Students are offered a variety 
of education?^l opportunities including formal and informal classes, 
intern programs and field trips. The students may or may not 
participate in any of these programs -- the choice is theirs. 
Selection and structuring of the classes and programs is totally 
dependent upon the desire of the students, their staff and their 
parents. Such classes as reading and writing (for primary children) 



literature, various inath courses, current events, psychology, 
youth and law, creative writing, cookery, photography, art, music, 
guitar, and five language classes are offered. In addition, a 
program of internships has been developed so that students can. 
work in the comnunity in various jobs which may be related to 
the vocation they intend to pursue. Further study is available 
through field trips in which students can learn about nature 
and themselves. 

The staff was selected by the parents and students. By 
dividing the appropriated salaries, the original allocated 
number of three certified staff members was expanded to six. 
With additional funding from SEA and other agencies (e.g. Urban 
Corps, Para-professionals), the staff including teachers, aides 
and volunteers has presently expanded to 14, rendering a student 
staff ratio of 5 students to 1 staff member during the first year. 

Although there is no principal (in name) of the Free School, 
a member of the staff was selected by his peers to serve as head 
teacher and is, therefore, responsible for the administration of 
the alternative. 

For the 1972-73 school year, an expansion occurred which can 
best be conveyed by discussing the comparative differences between 
1972-73 and the previous year. A comparative overview of Free 
School is made in the following dimensions: 

• Governance within the Free School Community - The Free 
School Governing Board is further developed conceptually and 
also appears to operate more efficiently. A constitution and 
by-laws have been established under the guidance of this yearns 
Free School director, Tony Morley. The concept of "community 
consensus" is still valued, but processes are far less apt to 
breakdown. Previously, the notion that everyone must be content 
with decisions of the Governing Board coupled with a general 
lack of mutually accepted procedures yielded few decisions. The 
system was dysfunctional as the failure of the group to generate 
decisions made it incumbent on individuals (i.e., head teacher) 
to do so. This was, in some ways, the antithesis of what people 
seemed to have in mind -- real community involvement in decision- 
making. The more functional Governing Board of this year may 
be attributed to any or all of the following: 

The Governing Board's accepting and learning 
from last year's experience. 

to A ,two-week communications-process-needs assessment 
"in-service" held before this school year began. 

9d The persistence of some in reminding others that 
some limits are functional, tasks must be accomplished, etc. 

The Comprehensive High School (7-12) - This junior and senior 
high school serves approximately 1150 students (levels 7-12). 



Although still in the process of developing and establishing its 
role as an SEA component, M-U High has already initiated some new 
programs. 

• As it is of paramount importance that M-U High be prepared 
to accommodate the educational demands of the incoming Southeast 
Alternative students, the seventh and eighth grade program is 
undergoing modifications that will allow for a smooth transition 
for all incoming students. 

0 An Off -Campus Career Exploration Program (O.C.L.E.) permits 
students to work in full-time non school -based activities and 
receive credit. 

• Project "AWARE", A Wilderness And Research Experience, is 

a one quarter alternative using an interdisciplinary-environmental ly 
oriented program. The areas of study cover navigation, field math, 
anthropology, geography, psychology, zoology, botany, astronorny 
and ecology. 

e A committee composed of M-U staff, students and parents 
jointly prepared a proposal entitled "Guide Groups" as an alterna- 
tive to home rooms which has been implemented to meet the need for 
more personal relationships between student, home and school. 

• A program entitled Adjusted Learning Environment (A.L.E.) 
was initiated for the *72-*73 school year after the completion of 
a summer pilot program. The program is designed for 7th graders 
needing close, individualized attention in further developing their 
abilities in basic academic and social skills. 

• In an attempt to provide students with an open learning 
environment, a program entitled Environmental Quarter is conducted 
with 24 seventh graders. The program consists of a three-hour 
session each morning which attempts to integrate English, social 
studies, math and science. 

With this abbreviated back drop, the evaluation design is develop- 
ed. Many novel programs and numerous significant themes have gone 
unmentioned or have been glossed over, but a basic comprehension of 
the SEA program should be possible from the previous descriptions. 

THE GENERAL DESIGN FOR EVALUATION AND DOCUMENTATION 

The early designers of experimental school evaluation and 
documentation efforts were surely aware of the special circumstances 
researchers would face. Traditional experimental design is precluded 
by the very nature of the project. There is only one SEA and only one 
Southeast community. There are, however, a number of internal and 
external comparison groups useful in developing a clear description 
and explanation of SEA. 
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The best way to conceptualize the evaluation and documentation 
of SEA is as a case study. This case study searches for themes of 
interest to federal decision-makers and practicing educators, A-^third 
group expected to benefit indirectly are those who are researching 
the process of schooling in field settings. 

At another level (cross site evaluation) a great many of these 
case studies (ESP projects) will be accumulated in order to draw con- 
clusions about the effects of comprehensive programming. Such an effort 
is only of interest to MET in that MET will cooperate, at times, in 
data collection for purposes of such macro-studies. The following plan, 
however, speaks only to the study of Southeast Alternatives, an Experi- 
mental schools project in Minneapolis, Minnesota. 

Viewing SEA as a rich setting to which one must bring a case 
study methodology, the questions become how best to deploy personnel, 
what data to collect and how to behave in the setting. The setting 
itself really dictates how to behave. The SEA experiment is to be 
respected as it is constituted by school personnel and protected from 
the influence of external evaluators. Hence non-interventionistic 
behavior is deemed imperative. Learning how to behave in the experi- 
mental setting has been no simple matter, however, for hundreds of 
hours are spent securing access to data and maintaining relationships. 

Personnel have been deployed in three basic ways. One contingent 
has conducted process observation of schools for purposes of document- 
ing the evolution of SEA. A second group has sought to estimate the 
impact of SEA on students, families, and structure through the use of 
traditional psychometric strategies. Finally, personnel are assigned 
to the administration of the evaluation including management of support 
services to the first two areas. 

During the first two years of the evaluation project, data has 
been collected through field observation, testing, interview, questionnaire, 
survey, critical incidents techniques, self-report and logical analysis. 
The way that these general techniques are used will be covered in 
detail as the plans and designs discussed in this paper. 

Specific Design Plan, January 1, 1974 - December 31, 1976 

The evaluation and documentation of SEA for the three year period, 
January 1, 1974 to December 31, 1976, will be structured around two 
separate but interacting components. These components are called The 
Anthropological Study and the Impact Study. These. two studies will be 
described in detail guided by the format of an overview, a description 
of the strategy employed, an outline of data to be collected, a state- 
ment of methods of data collection and a presentation of analyses to 
be performed. 

Questions of Interest 



Before getting into the two studies it might be useful to discuss 
briefly the possible usefulness of this massive evaluation and documentation 



effort. Below is a brief listing of potential benefits which 
accrue to NIE and the audiences it represents as a function of having 
funded this effort. 

0 The issue of the amount of time a child should spend in school 
has recently been raised in discussions concerning the cost of 
schooling. Within the natural setting of SEA, students spend different 
amounts of time in formal schooling. With the additional possibility 
of a four day school week, data will be available on the relationship 
of time in school to the process of schooling and school outcomes. 

0 The age at which a child begins formal education has been the 
subject of cross national studies (5) which tend to call into question 
much of the folklore on early beginnings. While most of the SEA 
children come under the school's care as of age 5, the advent of 
formal instruction differs across schools as does the intensity of 
instruction and level of expectation. Although we are talking about 
small numbers of students, we can follow these students for many 
years and gather information on all aspects of their life to address 
the issue under scrutiny. In addition considerable theoretical work (6) 
is underway which should provide a welcome backdrop for analysis. 

c It has been assumed for some time (and state guidelines are a 
result) that schools should be organized in classes and that classes 
should spend their time in certain prescribed ways; so many minutes 
for English, so many minutes for math etc. The schools of SEA 
have four different organizational patterns at the elementary level 
and at least two at the secondary level. There is much to be learned 
about the relationship of school organization and curriculum to the 
outcomes of schooling. 

d Affective learning has already become a "pop" expression, an 
unfortunate circumstance. Of all the things MET has studied or plans 
to study, the area of affect is the most interesting. Much should be 
learned from the multiple approach of measuring student affective 
behavior over time, deriving teacher intents in affective areas, 
identifying prevalent belief systems of families and observing the 
affective dimension of school life (e.g. order maintenance, teacher- 
student interactions, etc.) 

• Ranking near affective learning and apple pie is community 
involvement. In SEA five distinct forms of community involvement in 
the governance of schooling are evolving. In addition, the involve- 
ment of community at the program level and the tri -faceted community- 
school district-university relationship provides rich data for the 
anthropological study. 

• How much choice can be offered within the public school system, 
to whom can it be offered and what are the fiscal and psychological 
costs? SEA provides a setting par excellence for studying and 
answering this question. 
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t Can formative evaluation make a lasting difference? The forma- 
tive evaluation team of SEA has cast the above question as one of 
critical importance. The study of their formative evaluation strategy 
and the answer to the question is of tremendous significance to school 
districts across the country. 

00© And then there's the big question to be addressed at each site 
by each level of evaluation. What are the advantages and disadvantages 
of comprehensive programming or comprehensive change strategies when 
compared to piecemeal change strategies? 

THE ANTHROPOLOGICAL STUDY 

The impetus for a documentation study grew out of a general 
recognition of the sad history of educational project reporting. 
Too many projects have gone by the boards without documentation sufficient 
to allow others to judge for themselves what took place, the success 
of the project and its potential usefulness in other locations. The 
early conceptualization of the MET documentation study could be described 
as psycho-journalistic. Through many iterations of self-assessment, the 
documentation study has come to be seen more as an anthropological field 
study than psycho-journalistic reporting and the following plans and 
designs reflect the change in conceptualization. 

A. A General Overview of the Anthropological Study 

The purpose of the anthropological study is to prepare a description 
of the evolution of Southeast Alternatives and its component parts in 
as comprehensive a fashion as is possible. Such a complete description 
will have three general uses. First, it will serve to document (perceive, 
record, analyze and explain) what happens in Southeast Minneapolis such 
that those who wish to replicate such a program will have a detailed 
process record to work from. Secondly, the process documentation will 
serve to provide explanations for differences picked up in the impact 
study. Thirdly, the ethnographies of specific schools or subcultures 
studied will advance the state of the art of applying anthropological 
techniques to the study of education. 

In documenting the evolution of alternatives in SEA, MET will also 
be concerned with recapturing the history (to the extent possible) 
preceding the initial design for the SEA project. Consistent with the 
idea of an evolutionary process, MET will synthesize its information, 
findings, observations, and impressions at specific points in time 
in reports which can later be analyzed and integrated into documentation 
of the life cycle of the project. 

The demands of a comprehensive documentation task are great. It 
is not possible for MET to record everything that happens, yet at 
the same time, the MET plan for documentation has to be broad enough 
in scope to record significant events or characteristics of SEA 
respecting the comprehensive nature of the experiment. 
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B. The Anthropological Strategy 



During the first year of the project (1971-72), in contrast to 
methods generally employed by educational researchers, field workers 
entered the SEA system with few, if any, predesignated hypotheses. 
Indeed, the trained field worker generally prefers to begin with no 
specific hypotheses, for he expects hypotheses to emerge from an 
initial period of observing and interviewing. He trusts that his 
trained eyes and oars will help him discover what is relevant to the 
system which he is studying. He tends to be wary of formulations 
drawn from outside the immediate field (from studies of other sites, 
or from theoretical literature). 

Over the course of the first two years, different observers began 
with different, frameworks and came out with differing sets of empirical 
observations. Given the same organizing framework, however, various 
observers of one field situation — or readers of the field notes 
should come out with approximately the same conclusions. (4) 

One general characteristic of the first two years' field work was 
its temporal, developing character. This initial phase of SEA obser- 
vation yielded field notes that are somewhat general and scattered, 
because specific foci of observational attention and specific hypoth- 
eses had not yet crystallized and the observers were caught in a 
training gap due to the changing conceptualization of the task. 
Significant classes of persons and events have emerged from early form- 
ulations, however, and certain general issues have come into focus. 
Formulations must not, henceforth, be left only to the informal test 
of daily observation but must be subjected to more deliberate, con- 
trolled inquiry or pinpointing. 

As A. L. Strauss (7) has pointed out, the phase of "pinpointing 
— involves a hard look at one's understanding of the reality under 
investigation - understanding that is most critical to the emerging 
set of propositions." Such pinpointing of propositions will be 
especially marked near the end of the data gathering in the anthro- 
pological model to be employed, but has occurred in smaller measure 
for particular propositions along the course of the field observation 
to date. Hence an early anthropological strategy is pinpointing. 

A second important feature of the field model being developed is 
the nature of the propositions being sought. The outcome of the SEA 
observation will not be a few carefully tested hypotheses but rather 
a set of interrelated propositions. Strauss argues that "field work 
is well advanced when many apparently scattered observations are 
related to two or more propositional sets and these sets in turn are 
demonstrably and logically related to one another." The evidence 
gathered to support this model will be of several kinds: for example, 
interviews with teachers and students, observations concerning 
critical events, recordings of chronologically unfolding events which 
may develop over many weeks, and events precipitated in order to see 
what will happen in the light of what is already suspected or 
hypothesized. 
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Above alU the observer is an active instrument in the gathering 
of data. He does not sit passively in one spot, watching and listening, 
but uses a variety of tactics and instrumentation. Since evenings seem 
to be filled with endless meetings, the SEA process observer cannot settle 
for daylight hours only. He must get to the different places within the 
system (e.g., school, school classrooms). He may at times follow key 
persons through the course of their routines or conduct structured inter- 
views for such behavior. In fact, it is expected that participant 
observation, unobtrusive measures, interviews and a whole host of 
techniques will be used to elicit a great quantity of rich naturalistic 
data. The question becomes one of focus, "What data will be collected 
and how will it be analyzed?" 

C. Data to be Collected 

The data to be collected in the anthropological study will, for 
the most part, be field notes and interviews recorded in chronological 
sequence. The first category* shakedown has already occurred and the 
resulting categories now serve as an internalized guide for field 
notes with a corresponding set of codes. The categories can be viewed 
as data bins to be filled. MET anthropologists will also be fully 
acquainted with the type of unobtrusive data that is often available 
in a field setting. 

D. Method of Data Collection 

The major field method employed is based on what Strauss (7) 
describes as "theoretical sampling" • Theoretical sampling is carried 
out to discover categories and their properties and to suggest inter- 
relationships forming a holistic theory. Categories can be judged, 
according to Strauss, by their width and diversity. 

MET will pattern its work in the theoretical sampling area after 
the model established by Barney Glaser and Anselm Strauss (3) in a 
field study on dying. In this study, to oversimplify, hospital locations 
were allotted observation time based on their theoretical significance. 
A setting was said to have theoretical significance, in the estimation 
of this writer, if it provided a conceptual comparison group. 

Participant observation remains a major method of data collection. 
"Participant observation is a conmitment to adopt a perspective of 
those studied by sharing in their day-to-dav experiences." (2) In 
participant observation the observer gathers data bv participating 
in the daily activities of the group he studies. Participant obser- 
vation will be used as an on site data collection strategy to develop 

* These categories include a listing of the major dimensions of the 
SEA project followed by a series of questions designed to point out 
the relevant aspects of the dimension. Prevalent belief systems, 
choice-making, governance and planning are four general category 
dimensions. 
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a data base from which strategic ethnographies can be developed. 
Once field workers are adequately grounded, however, other sites 
within Minneapolis and alternative schools outside of Minneapolis 
will be observed from the standpoint of developing notions about the 
unique and the common characteristics of SEA. That is, MET predicts 
theoretically significant samples will be outside of SEA. 

Participant observers will be assigned to one or more of five 
settings. These include the following: 

0 The K-12 free school and the K-6 open school. 

0 The K-6 continuous progress school and the K-6 contemporary 
school . 

• The 7-12 hiqh school. 

A The SEA support staff center and the Southeast Minneapolis 
community. 

• Locations outside of SEA including other classrooms in 
Minneapolis and locations selected for their theoretical valuo 
outside of SEA (e.g. Venture School, Palo Alto). 

Each of these assignments is considered a component in later staffing 
charts. 

Event analysis and network analysis are also methods of data 
collection to be used in the MET Anthropological Study. These tech- 
niques will provide a history of events and a network of individuals 
involved in events. In addition the Rachomon technique will be 
employed in describing an event from several vantage points. Life 
histories also provide an indepth perspective of importance to any 
detailed field study and will be a method utilized often. 

E. Assuring the Dependability of Information 

Observational methods are especially vulnerable to the fallibilities 
of human perceivers. The consequences of this fact for process observation 
methodology can be sizeable. The points to be considered in "calibrating'* 
field observers must be conveyed to them, typically during training. The 
conduct of training can be crucial in the determination of the impact 
that observational, and other errors will have in the analysis. It is 
during training that events become more familiar and definitions of categories 
will become more precise. As the observer gains more experience with a 
given system, former associations among dimensions may have less effect, 
with the result that evaluative responses give way to studied responses. 

It can be expected that when observers first start to learn the* 
definitions of categories and the rationale for the system, their 
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reaction will typically be one of concern with the apparent complexity 

of the* system. This is one of the reasons it is especially important 

to provide information about how the observational categories were 

built* so that. the observer can be instrumental in deriving other 

sets of categories and definitions. Written protocols have been valuable 

because they permit trainees to ponder interpretations without the pressures 

of immediate decisions. 

Two additional subtle issues pervade training. One concerns the 
frustrations that occur when an act (behavior) is missed or placed 
in the wrong category. A good observer must learn tolerance for 
these errors. One means to promote such tolerance is to acquire 
an ability to inhibit all but the present context of acts and to 
avoid jamming incoming stimuli with internal reflections. 

A second, and potentially more important issue, concerns criteria 
for the correctness of categorization. Many observers see categorization 
as largely a common sense activity, and they may be reluctant to admit 
that they make mistakes and that someone else has more "common sense". 
Persons in training can be expected to strongly defend their own 
interpretations. The best means to head off these confrontations 
is to maintain permissive training orientations and use external criteria 
of "correctness". 



Methods can affect some observer errors; training can affect 
others. Judicioiis choices of method and training techniques can help 
the investigator retain the advantage of the human instrument without 
all the attendant errors. But such an outcome is difficult to come by 
since the observer's task is often not very different from that of a 
person in everyday life. 

F. Analyses To Be Reformed 

As noted previously, Appendix A contains the grounded categories 
developed to date for the Anthropological Study. These are the "bins" 
into which data is poured and from which come brief descriptive summaries. 
The interrelationship between category descriptions and general grounded 
theories becomes a source of data for explanations . 

It should be noted that in the conduct of participant observation 
a number of analytic techniques are used. The observer constantly 
updates and redesigns his effort as he uncovers new data. This continuing 
analysis leaves field notes undigested for only short periods of time 
and should result in tentative conclusions at the end of the field work 
period. — 

The end of field work is not the beginning of analysis but only 
a continuation. At this point, however, methods of summary must be 
chosen. Conclusions are often supported bv anecdotes, illustrations 
and abbreviated life history material. Verifications are made when an 
independent observer reviews the field notes to assess the relationship 
of data to conclusions. 
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THE IMPACT STUDY 



A. Impact-General Overview 

The purpose of the Impact Study is to assess the effect of SEA 
on the schools, the students and their families, and the community. 
As a dynamic experiment in alternative schooling, SEA undoubtedly 
affects many of those participating in the project. The task of the 
Impact Study is to learn which of the participants in SEA are affected, 
how they are affected, and if the effects are significant. Data from 
the Impact Study should form an integral part of the information 
necessary to judge in some ways if alternative schools are worth the 
effort. In studying this experimental approach to education, the 
greatest emphasis will be placed on the impact of SEA on students. In 
assaying student impact, student academic performance, attitudes 
towards school, involvement and use of choice will be reviewed. It 
will be observed whether the opportunities for choice are actually used 
and what effect the alternatives have on learning and the happiness 
of the pupils. The attitudes of families toward SEA will be carefully 
studied. Current evidence suggests that to be effective a school 
system must reflect and respond to the attitudes and opinions of the 
immediate community. With this hypothesis in mind, the views of the 
community will be continually monitored. Throughout the various Impact 
Studies, special attention will be directed toward choice and how well it 
operates in SEA, because the opportunity for choice has been heralded as 
the most important and distinctive feature of SEA. Within the schools 
of Southeast, the opinions of the staff will be surveyed, because no 
approach to education can hope to be successful without staff approval 
and support. Possible staffing benefits or problems under an alternative 
system will also be noted. Data on impact will be gathered for the 
duration of SEA, and the data will be standardized and organized to the 
extent that it can speak reliably on changes that may take place. At 
the same time, there will be no hesitation to collect, in a systematic 
way, qualitative data that give important insights into the operation 
of Southeast Alternatives. 

B. Impact Strategy 

The strategy for collecting Impact data will be eclectic. A 
number of different methods are to be used to obtain student impact 
data. Objectives based tests have been constructed to gather information 
on the math and reading performance of elementary students the objectives 
being those common to all SEA schools. Some normative testing results 
will be available for elementary students from the Testing Program of the 
Minneapolis Public Schools. Unfortunately, MET cannot be certain that 
Minneapolis will continue to require tests of elementary students for the 
duration of SEA. Affective information from elementary students will 
be obtained by the MET affective survey (part of the MET testing program) 
and by "mini -interviews". The Minneapolis Testing Program does not 
require secondary students to take normative tests in various subject 
matter areas on a consistent schedule and MET has no plans for testing 
basic skills on the secondary level. Affective data on secondary students 
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is available from the Minneapolis Public Schools in the form of data 
from two opinion questionnaires (SOQ and QUESTA); "mini -interviews" 
have also been given to the secondary students. 

Case studies represent a method of collecting data on both students 
and their families. The case studies involve extensive interviews of 
the students and families, and they occur at the beginning and towards 
the end of each school year. Contact with the case study families 
throughout the year is encouraged, and presently such contact is 
facilitated by special telephone recording equipment referred to as 
the MET "hot line". 

Contingency case studies are used as events or topics of interest 
arise during the year. The events or topics are selected for their 
potential in illuminating and illustrating the workings of SEA. The 
participants in the events are interviewed in detail. 

Surveys are used as a way of getting at community attitudes on 
dimensions and topics of interest to SEA. A staff survey is used to 
monitor current opinions and feelings of the SEA staff. Other sources 
of data include special studies by SEA's Level I evaluation team. An 
important example of such special studies is the Level I Mobility 
Study which reviewed the patterns of choice in SEA. 

C. Data To Be Collected 

Getting down to specifics, a large quantity of data has been 
collected and will be collected over the five year life of SEA. The 
following is a list of the major chunks of data which have been 
collected, or will be collected. 

e MET testing data. Data from the fall '72 MET testing program is 
now available. Raw data from the spring '73 MET testing program will 
be available on May 14, 1973. In future years MET testing will take 
place each spring. The tests include primary and intermediate reading 
tests, primary and intermediate mathematics tests, and primary and 
intermediate affective surveys. The math and reading tests are 
based on common objectives of the SEA schools. 

9 Minneapolis Testing Program. Data from the Minneapolis Testing 
Program will be collected to provide normative test information and 
background data. On the elementary level, different versions and 
supplements of the Gates-MacGinitie Reading Tests and the Iowa Tests 
of Basic Skills are available. The Lorge-Thorndike Intelligence Test 
is also available. It is not clear at this time what Minneapolis test 
results will be obtained, because the Minneapolis Testing Program is 
changing and we cannot tell which tests will continue for the life of 
SEA. On the secondary level, the results of two opinion questionnaires 
— Student Opinion Questionnaire (SOQ) and Questionnaire I'or Students, 
Teachers and Administration (QUESTA) — will be obtained. 
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• Case Studies. Interviews with the families selected for case 
studies have been conducted twice. The third interview, the spring 
'73 Case Study, is now in progress. Case study interviews will be 
carried out in the fall and spring of each remaining year of SEA. The 
interviewing is done by two people, both of whom have social work 
experience. 

0 r 'tingency case studies will be done whenever the situation or 
potential /r, formation merits the effort. One contingency study of 
M.U. students with severe reading difficulties has already been 
done. Contingency case studies of students transferring, during the 
school year, from one Southeast alternative to another, is slated for 
May, 1973. 

0 "Mini -interviews" of students are now being conducted in the 
elementary schools and the high school. The first experimental 
"mini-interview" deals with choice in the schools; the interview was 
developed by MET. Continuation or expansion of the mini-interviews 
will depend on a complete analysis. 

t Community Surveys. Data is now available from two community 
surveys; partly analyzed data from the third community survey should 
be available by the middle of May, 1973. MET intends to continue 
yearly community surveys, hopefully with the continued cooperation 
of sea's Level I. In an attempt to get more complete representation 
of the SEA community than this year's mail-out survey permits, a 
door-to-door survey is being considered for next year. The next 
survey will be conducted in the spring of 1974. 

D. Methods of Data Collection 

Some of the methods of collecting data are already apparent from 
the foregoing discussion. Further specifics will be provided where 
it is felt they are needed. 

e MET testing data. The match and reading tests designed by MET 
are administered by the teachers in the SEA schools. The tests are 
dropped off and picked up on a day-to-day basis. The MET Affective 
Survey is administered by MET personnel. This spring, starting with 
the week of April 23, one week, with definite dates for each school, 
has been assigned for the reading tests, the math tests, and the 
affective tests. The tests will be scored and analyzed immediately 
and a similar procedure will be used in the spring of 1974 and in 
the following years. 

0 Minneapolis Testing Program. A preliminary analysis of this 
school year's SOQ data has already been obtained, and all available 
data from both the SOQ and QUESTA will be obtained from^the Educational 
Services Department of the Minneapolis Public Schools when they are 
ready. As soon as a reasonable decision can be made about which data 
will be useful, elementary school test data will be requested from the 
Educational Services Department of the Minneapolis Public Schools. 
The data from the elementary schools will be requested this spring. 
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f Family Case Studies. Information from the family case studies 
is collected on a day-to-day basis. Interview dates are set by the 
case study workers and confirmed by the secretaries. This routine, 
coupled with persi stance has yielded close to complete returns. When 
a case study family proves to be impossible to contact, a careful 
decision is. made about retaining them in the case study samr''^; 
fortunately, this has happened only once. 

9 Contingency case studies are not whimsical, but tu9:y are done 
by different members of the MET team as potentially informative 
events or situations are identified. 

9 Data for the "mini -interviews" on choice is now being collected 
by the process observers. This data should be in by the end of the 
first week in May. 

0 Community surveys. Data from the first two community surveys 
is available at ARIES. The Fall 71 Survey — a joint Level I and 
Level II effort — has already been analyzed. The Summer Survey, 
conducted by the University of Minnesota, for Level I resulted in a 
report which gives an extensive analysis. The '73 Parent Survey 
was another Level I and Level II effort. Because Level II was 
largely responsible for developing the survey, it was decided that 
the data collection and preliminary analysis would be the responsi- 
bility of Level I. As of Monday, April 23, 617 surveys had been 
returned, and 3 Level I people have been coding the open-ended 
questions and keypunching the closed questions. The cutoff date for 
accepting surveys is April 27 and the keypunching and preliminary 
analysis is to be completed by May 15 at which time the data, includ- 
ing keypunch cards will be available to MET for further analysis. 

E. Dependability of Data 

The dependability of the data collected is an increasing concern 
of MET's. Ever.v effort is being made to assess, and where necessary, 
to increase the reliability of the information obtained by MET. 

© MET testing data. During the late spring of '73 and early 
summer checks of the reliability of the MET primary and intermediate 
reading and math tests are to be run using coefficient alpha as the 
measure of reliability. Changes necessary to increase the reliability 
of the MET tests will be made during the summer and fall of 1973. 

0 Minneapolis Testing Program. The Minneapolis Testing Program 
consists largely of familiar normative tests which are encrusted with 
reliability data. 

0 Case studies. The reliability of the case study data will always 
be a concern. However, the insights and the richness which the case 
studies are capable of providing, make the reliability risks worth 
taking. Within the next month. Dr. Buelah Compton from the University 
of Minnesota will be reviewing the entire case study procedure and 
verifying the conclusions drawn from case study data. A similar 
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verification procedure will follow each round of case study interviews. 
Consistency in instruments and procedure has been attempted, but 
changes in our sample of 60 families is beyond our control. With the 
help of school social workers and other people familiar with Southeast 
Minneapolis families, families representative of the Southeast community 
have replenished our sample when necessary. It appears that a 
substantial portion of our case study sample will be available for the 
life of SFA. 

0 Community surveys. The questions and format of the 1973 Parent 
Survey have been reviewed by an expert from the University of Minnesota 
and found to be a substantial improvement over the 1971 survey. Many 
of the current survey questions will be retained in their present form 
for future surveys. In spite of efforts to reach all strata of South- 
east MinneapoliSs the 1973 survey returns are biased as are all mail -in 
surveys. We are considering a sample of Southeast families and door- 
to-door interviewing to avoid this problem next year. 

F. Analysis* 

During the summer of '73 a major part of the work of the Impact 
Team will be further analysis of available data. MET's effort will 
be to follow changes occurring in student and community behavior 
during the course of SEA. 

0 MET testing data. The fall '72 testing data has been reduced 
by a preliminary analysis. For math and reading tests, performance 
on all items by school and age are available. A preliminary analysis 
of item difficulty and tests of internal consistency will be carried 
out this summer. In addition to coefficient alpha an attempt will be 
made to test the homogeneity of items within objectives. On the 
reading tests 2 SOBE-R items were often used for each objective — these 
items having been selected from 5 SOBE-R items keyed to the same 
objective. The correlations of the items testing the same objective 
will be examined, and the correlations for similar math items will 
also be obtained. Similar analysis will be made for the spring '73 
results. The performance of various subgroups within and between 
school may be compared as they are suggested by the data. A prelim- 
inary analysis of the affective results has broken the data down by 
school for questions dealing with general categories of our affective 
taxonomy. These results have been further analyzed into responses by 
primary and intermediate students. Similar analyses of the spring test 
data will be made. A potential source of information on trends in SEA 
is contained in records of the SEA school's stated objeuiiives in math, 
reading, and the affective domain. Careful note will bt made of any 
changes in our list of school objectives. 



* The section, "Questions of Interest" provides the rationale for 
analysis. That is, it is the purpose of analysis to shed light on those 
questions. 
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0 Minneapolis Testing Program. Only the SOQ and QUESTA data will 
be definitely used. Data from these opinion questionnaires will be 
used in conjunction with information obtained by our process observers, 
case studies and community survey to corranent on choice, involvement 
and attitude towards school. Elementary normative tests will be used 
to the extent they can give backgrounu information on the way Southeast 
schools were before SEA, and to the extent they can tell us about the 
current distribution of intelligence across SEA schools. Minneapolis 
test information will also be used when it provides further information 
on our own testing effort. 

0 Case studies. Case studies will be analyzed p-^rti arly for 
the information they give about choice, student and parent involvement, 
and satisfaction with schooling. It is expected that the case studies 
win add a great deal of richness to the more statistical data from 
other sources. 

© Community surveys. The community surveys will be viewed for the 
information they can yield on general attitudes towards education, 
specific attitudes towards SEA and its problems, specific feeling about 
individual schools and their performance, as well as information on 
choice and involvement. 

A general strategy used throughout Impact analyses will be that 
of approaching a question with different instruments and modes of 
attack. Choice, for example, will be investigated in a number of 
different ways. Information on choice comes from the Case Studies, 
the Community Surveys, Level I Mobility Study, SOQ and QUESTA, the 
mini-interviews and observations of the process observers. If there 
is substantial agreement among several levels of observation on a 
topic such as choice, belief in the findings will increase accordingly. 
For example, the school children state certain opinions in the mini- 
interviews, which are reinforced by the observations of the process 
observer, and written opinions on SOQ, QUESTA or MET's affective 
tests, as well as case study information, a good deal of confidence 
can be placed in the findings. 

Limitations of the Anthropological and Impact Studies 

There are always natural limitations, measurement considerations, 
conceptual loopholes and the like which limit research. Some of the 
problems, constraints and issues that MET is cognizant of are listed 
below. 

Internal and external validity are useful concepts as defined 
by Webb and others (8) in their book on Unobtrusive Measures. Webb 
notes that internal validity asks whether a difference exists at all 
in any given comparison or whether an apparent difference can be 
explained away as a measurement artifact. External validity is the 
problem of interpreting a difference and generalizing to other 
populations. Webb goes on to note a set of sources of invalidity 
which will be applied to the SEA setting. 
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• Awareness of Being Tested , The students, staff and parents 
of SEA are fully aware of the MET testing program among other things. 
In specialized cases this could result in a positive or negative 
"guinea pig'' effect. 

0 Role Selection . Webb notes that people have many roles to play 
and may use the role they feel the experimenter will be most pleased 
with during observation or interview sessions. Such an artifact 
should be evident after five years ^ research in many settings. 

© Measurement as a Change Agent . The actual process of measure- 
ment can serve as a change agent in certain circumstances. In the 
survey work of MET care on this issue is of considerable importance. 

3 Response Sets . It has been shown that subjects will more 
frequently endorse a statement than disagree with its opposite. When 
working with the early elementary school students of SEA, especially 
with the affective test, care must be taken to provide a neutral or 
accepting test environment. This means among other things replacing 
the teacher with a trained test administrator and assuring the students 
that teachers will not be apprised of their answers. 

c Interviewer Effects , The religion and race of an interviewer 
may bias results obtained. Some of these biases can be controlled 
for, rotated out or lived through while others are quite subtle and 
need considerable study. Frequent tri angulation of major themes 
may provide one way to deal with the ever present biases of face to 
face interviewing. 

0 Change in the Research Instrument . An interviewer may tire and 
change over time and an observer may be influenced by the environment 
and modify his observation strategy. Adequate training, and frequent 
reliability checks can reduce the impact of this cause of invalidity. 

e Population Restriction . Many methods automatically limit the 
universe to be studied or a method by universe interaction often 
exists. A telephone survey will only generalize to those who have 
phones and are home when the calls are made. A mail survey will only 
generalize to those who live at the address noted and take the time 
to complete and return the survey. 

0 Population Stability . As noted by Webb, a common failure in 
survey research where multiple measures are taken over time is to 
forget to take into account the mobility of the population. One 
solution would be to use only cohorts for discussion of longitudinal 
change. 

Some other items specific to the SEA that might serve to invalidate 
the findings are as follows: 

9 The Outsider Effect . Evaluators are relegated to the role of 
outsider for their task is of no immediate benefit to those under 
study and they are not typically accountable to those they evaluate. 



22 



Depending on the ability of the outsider to keep good relations in a 
tradeoff situation where he has bubble gum cards while his subject 
has $100,00 bills, the outsider effect can be important, 

0 The Role and Level Effect , Ambiguity in the relationship and 
responsibilities of various levels in multilevel research can be 
undesirable. An important first step in multilevel research must 
be defining who's doing what to whom for what purpose and under whose 
authority. 

t The Missing Base Line Trick . In many social systems records are 
not kept in a manner consistent enough to provide a researcher with ample 
baseline data on the system prior to the intervention. This is a major 
problem for MET to solve in SEA. 

0 " Au Natural " Research . Taking a setting as it is and attempting 
to understand and explain what takes place there is somewhat novel in 
the history of educational research. One way of dealing with this 
potential source of invalidity is to seek out sociologists and 
anthropologists who are quite comfortable with a design of this sort, 

0 The Complications that Stem from Choice . It would be hard to 
design a system of schooling that would make it any harder to attribute 
affects to some aspect of the program. During the first year of SEA 
30% of the students attended other than their neighborhood school and 
these students had distinct characteristics that made them anything 
but a representable sample. Hence a major aspect of the program 
design, choice, places severe constraints on the evaluation design, 

Campbell (1) also detailed a number of factors that affect internal 
and external validity. Five factors not previously discussed but of 
some importance in the SEA setting are noted below: 

e History. Much happens between two measures of the same trait 
of class of traits other than the treatment designed to or expected 
to bring about a change. In fact factors of not being monitored may 
be substantially influencing. 

e Maturation. One must be careful to account for the passage of 
time prior to concluding that some treatment had brought about a change, 

• Testing. In the laboratory research the act of giving a test 
has been shown to have a significant effect on scores achieved during 
the second measurement period. 

t Experimental Mortality. The mortality factor was previously 
discussed and it is also important to watch for an uneven mortality 
rate over treatment groups, (e.g.. Do you lose students from school X 
more often than school Y and are the two groups of leaving students 
from the same or different populations?) 
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0 Multiple Treatment Interference. 

An item relevant to external validity is the situation where 
many treatments affect the same subjects. This is an important con- 
sideration but an over simplification, I'm afraid, of the truly 
complex treatment situation found in SEA. 

The evaluation and documentation of SEA is a complex undertaking. 
The reader must realize that any one of the above could dramat^ically 
affect the results of MET observations. However, as often noted 
above, steps have been and will be taken to reduce the influence of 
these potentially troublesome factors. 

In summary, an innovative project has been described as has the 
evaluation designed to measure its effects and document its evolution. 
One aspect of the evaluation utilizes an uncommon anthropological 
approach which is discussed in detail. The limitations of such a 
procedure are discussed in general terms. 
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